1. Field of the Invention
The present invention relates to an image processing method for providing a desired, optimum dynamic range to a digital color still picture that has been captured by a digital camera, a monitoring camera, etc.
2. Description of the Related Art
As a method for correcting brightness information, color information, etc., of a digital color still image, a retinex method taking a retina of an eye-ball as a model is proposed, in which a brightness value of an image with a biased dynamic range (dynamic characteristic) is corrected based on local information of the image. In such a retinex method, the visibility of an image can be improved by correcting a low brightness value resulting in a darker portion in the image to a high brightness value, and a high brightness value resulting in a brighter portion in the image to a low brightness value.
As examples of such a retinex method, the single scale retinex method (Daniel J. Jobson, et al., “Properties and Performance of a Center/Surround Retinex,” IEEE Trans. on Image Processing vol. 6, pp. 451-462, March 1997) and the color restoration multiscale retinex method (Daniel J. Jobson, et al., “A Multiscale Retinex for Bridging the Gap Between Color Image and the Human Observation of Scenes,” IEEE Trans. on Image Processing vol. 6, pp. 956-976, July 1997) are proposed.
In the single scale retinex method, a spectrum component Ii(x,y) (i=1, 2, . . . ) of a target pixel I(x,y) in an original image is corrected by a surround function F(x,y) obtained from image information of the surround pixels. The result Ri of retinex processing is shown by expression (1):Ri=log(Ii(x,y))−log[F(x,y)*Ii(x,y)]  (1)where “*” denotes a convolution operation.
In this case, the surround function F(x,y) is a function which is based on the image information of the surround pixels with respect to the target pixel I(x,y), which is represented by expression (2):                               F          ⁡                      (                          x              ,              y                        )                          =                  Ke                                    -                              r                2                                      /                          c              2                                                          (        2        )            where r denotes a distance (r2=x2+y2) between the target pixel and the surround pixel, and c is a scaling variable. Furthermore, K is a scalar coefficient, which is set so that ∫∫F(x,y)dxdy=1 is satisfied, i.e., the total value of the surround function is 1.
As described above, in the single scale retinex method, the spectrum band components of the target pixel are retinex-processed based on the surround function. On the other hand, in the color restoration multiscale retinex (MSR) method, each of the spectrum band components, Ii, of the target pixel is retinex-processed by N types of scales. In this case, the result of the single retinex process for each of the N scales is multiplied by a weighting factor ωn, and the multiplication results with respect to all the scales are totaled. The result of the multiscale retinex (MSR) processing, RMSRi, is represented by expression (3):                               R                      MSR            i                          =                              ∑                          n              =              1                        N                    ⁢                                           ⁢                                    ω              n                        ⁢                          R              ni                                                          (        3        )            
As described above, in the MSR processing, the retinex processing is carried out for each spectrum band component of an image. Therefore, the ratio of the spectrum band components in the image is changed, causing a color shift. According to a conventional technique, such a color shift is corrected by a color restoration function. In this case, at first, a color space of an original image is converted into a relative color space by expression (4):                                           I            i            ′                    ⁡                      (                          x              ,              y                        )                          =                                            I              i                        ⁡                          (                              x                ,                y                            )                                /                                    ∑                              i                =                1                            S                        ⁢                                                   ⁢                                          I                i                            ⁡                              (                                  x                  ,                  y                                )                                                                        (        4        )            
In expression (4), I′i(x,y) denotes a color component of the original image, and S denotes the number of dimensions of the color space. When the three primary colors (R, G, B) are used, S=3. With such a color component, I′i(x,y), the color correction coefficient Ci(x,y) is obtained by expression (5):Ci(x,y)=f[I′i(x,y)]  (5)
This color correction coefficient Ci(x,y) can be rewritten as in expression (6):                                                                                           C                  i                                ⁡                                  (                                      x                    ,                    y                                    )                                            =                            ⁢                              βlog                ⁡                                  [                                      α                    ⁢                                                                                   ⁢                                                                  I                        i                        ′                                            ⁡                                              (                                                  x                          ,                          y                                                )                                                                              ]                                                                                                        =                            ⁢                              β                ⁢                                  {                                                            log                      ⁡                                              [                                                  α                          ⁢                                                                                                           ⁢                                                                                    I                              i                                                        ⁡                                                          (                                                              x                                ,                                y                                                            )                                                                                                      ]                                                              -                                          log                      ⁡                                              [                                                                              ∑                                                          i                              =                              1                                                        S                                                    ⁢                                                                                                           ⁢                                                                                    I                              i                                                        ⁡                                                          (                                                              x                                ,                                y                                                            )                                                                                                      ]                                                                              }                                                                                        (        6        )            where α and β are constants.
The result of the MSR processing, RMSRi, obtained by expression (3) is corrected by the color correction coefficient Ci(x,y) of expression (6), thereby obtaining the result of the color correction multiscale retinex processing, RMSRCRi(x,y), as shown in expression (7):RMSRCRi(x,y)=Ci(x,y)RMSRi(x,y)  (7)
The result of the color correction multiscale retinex processing, RMSRCRi(x,y), can be rewritten as in expression (8):                                           R                          MSRCR              i                                ⁡                      (                          x              ,              y                        )                          =                  G          ⁡                      [                                                                                C                    i                                    ⁡                                      (                                          x                      ,                      y                                        )                                                  ⁢                                                      ∑                                          n                      =                      1                                        N                                    ⁢                                                                           ⁢                                                            ω                      n                                        ⁢                                          {                                                                        log                          ⁢                                                                                                           ⁢                                                                                    I                              i                                                        ⁡                                                          (                                                              x                                ,                                y                                                            )                                                                                                      -                                                  log                          ⁡                                                      [                                                                                                                            I                                  i                                                                ⁡                                                                  (                                                                      x                                    ,                                    y                                                                    )                                                                                            *                                                                                                F                                  n                                                                ⁡                                                                  (                                                                      x                                    ,                                    y                                                                    )                                                                                                                      ]                                                                                              }                                                                                  +              b                        ]                                              (        8        )            where G and b are constants.
As described above, the result of the retinex processing that is carried out for each of the spectrum band components of the original image is corrected based on the color correction coefficient, thereby preventing the color shift caused by the change in the ratio of spectrum band components.
Such a color correction multiscale retinex method is applied to a medical photograph, an aerial photograph, or the like. With this method, images, patterns, or the like, which cannot be perceived by a human eye, are corrected so as to be perceived by the human eye. However, such a correction does not faithfully reproduce an object as an image. Thus, the color correction multiscale retinex method is not necessarily a suitable processing method for a monitoring camera, a digital still camera, or the like.
Furthermore, the multiscale retinex method requires a product-sum operation which is necessary for a log operation, an exponentiation operation, and a convolution operation. Therefore, as the number of surround pixels to be referred to increases, the calculation amount also increases. Moreover, a memory with a large capacity is required for storing the information of the two-dimensional surround pixels, the result of intermediate processes, etc. Accordingly, when a device using the retinex method is realized, the economical efficiency thereof might be deteriorated. In addition, since the number of accesses to a memory that stores operation values increases, there occurs a possibility that the processing time might increase.
Especially in recent years, digital cameras with high resolutions in which the number of subjective pixels is one or two megapixels have been developed. In such a digital camera, the time required for the MSR processing further increases.